Z 


JYVASKYLAN YLIOPISTO 
UNIVERSITY OF JYVASKYLA 


Learning outcomes and learners’ impressions 
of parallel and monolingual concordancers 


June Ruivivar! and Cynthia Lapierre’ 


Abstract. Monolingual and parallel concordancers have both been found to benefit 
second-language (L2) grammatical development. However, the relative benefits of 
these two concordancer types remains unclear. The present study compares learning 
outcomes and learners’ perceptions of a monolingual English (Corpus of Contemporary 
American English, COCA) and a French-English parallel concordancer (Tradooit), 
using verb-preposition collocations as a target feature. Students in an advanced English 
as a second language (ESL) course completed three concordancing activities where they 
used either Tradooit (for French L1 speakers) or COCA (for other L1s) to formulate 
rules for challenging verb-preposition collocations (e.g. arrive in/at). Pre- and post-tests 
showed significant learning gains for both groups, which were maintained in delayed 
post-tests; however, perception questionnaires showed that Tradooit was perceived as 
easier and more useful for language learning. We suggest that these differences may 
be due to the type of cognitive work involved in L1-L2 comparison versus L2 pattern- 
finding, but that these two processes may both lead to noticing and learning. 
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1. Introduction 


Data-Driven Learning (DDL), the use of authentic language samples to examine 
patterns of language use, is increasingly recognized for its affordances in grammar 
instruction. In particular, it can draw learners’ attention to forms that they may 
otherwise overlook (Moon & Oh, 2018) and involve a greater cognitive load, 
which may help learners internalize language items (Herron & Tomasello, 1992; 
Smart, 2014). 
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DDL activities are typically carried out using either monolingual or parallel 
concordancers. Monolingual concordancers draw on samples in one language; for 
example, a learner may find that throw, when used with its common collocate away, 
means fo dispose of. Parallel concordancers, on the other hand, provide examples 
from two or more languages and are often used for L1-L2 comparisons. To use the 
same example, a French-speaking learner might find that jeter, when used in the 
sense to dispose of, translates to throw away, not throw, in English. 


Research suggests that both types of concordancers result in learning gains and are 
well received by learners (e.g. Huang, 2014 for monolingual, and Gao, 2011 for 
parallel). However, to our knowledge, no published research to date has directly 
compared these two types. It is not clear, then, whether the documented benefits of 
monolingual and parallel concordancing are attributable to DDL itself, or to features 
unique to each type, such as learners’ preferences. The present study attempts to 
address this gap by comparing students’ learning gains from, and perceptions 
of, a monolingual English (Corpus of Contemporary American English, COCA) 
and parallel French-English concordancer (Tradooit). Specifically, we wanted to 
compare: 


¢ the effects of a parallel and monolingual concordancer on leamers’ 
recognition and productive knowledge of verb-preposition collocations, 
and 


¢ learners’ perceptions of each concordancer’s ease of use, usefulness for 
coursework, and usefulness for general language improvement 


Verb-preposition collocations (e.g. insist on, wait for) were chosen as a target 
feature because it was not part of the students’ course syllabus, making it possible 
to isolate the effects of DDL from classroom instruction. Although students may 
have previously encountered the form, pre-test results suggest that it continues to 
be a challenge for them. 


2. Method 


2.1. Participants and context 


Participants were 23 students, aged 17 to 40, in an advanced ESL course taught by 
the second author. The course covered advanced grammar, vocabulary, and source- 
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based writing, and consisted of two 2.5-hour sessions per week over 13 weeks. 
Students spoke a variety of languages, including Arabic, Chinese, French, Hebrew, 
and Spanish. These languages were used to assign them to concordancers for 
the study. Ten proficient or native French speakers were assigned to the parallel 
concordancer, Tradooit; the remaining 13 were assigned to the monolingual 
concordancer, COCA. 


2.2. Procedure 


2.2.1. Concordance training 


Each group received a 1.5-hour training session on their assigned concordancer. 
Students were introduced to corpora and concordancing, then provided with 
instructions on using the concordancer, with focus on collocations. They were then 
guided through practice activities where they looked up a list of expressions on the 
concordancer. They also performed a pilot task (not included in the analysis) to 
ensure that they were familiar with the concordancers before collecting data. 


2.2.2. Treatment 


Throughout the semester, students completed three guided induction activities where 
they corrected a set of sentences by looking up expressions on the concordancer, 
formulated a hypothesis about the underlying rule, and provided two concordance 
lines illustrating the rule. 


Each assignment included three to four of the 10 target verb-preposition collocations, 
and three grammar items from the previous unit to serve as distractors. These were 
given as homework and graded on completion rather than accuracy. 


2.2.3. Instruments 


Learning outcomes were measured with pre-, post-, and delayed post-tests 
measuring recognition (multiple choice) and controlled production (gap-fill). The 
tests consisted of the ten target items and ten distractors from the previous unit. 


Perceptions were measured using an end-of-course questionnaire asking them to 
rate three items on a ten-point scale: how easy the concordancer was to use, how 
useful it was for the course, and how useful it was for learning English in general. 
There was also space for them to write comments. 
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3. Results and discussion 


3.1. Learning outcomes 


To compare learning outcomes, paired-samples f-tests were performed between 
the pre- and post-tests, and between the post- and delayed post-tests. Table | shows 
an increase in mean recognition scores for both groups, which was maintained on 
the delayed post-test. Pre- to post-test gains were significant for both the COCA, 
t(12)=7.63, p<.001, and Tradooit groups, 4(9)=9.22, p<.001, but not significant 
between post- and delayed post-tests. To compare improvement between groups, 
we conducted independent-samples t-tests between the mean difference in scores 
between the pre- and post-tests. We found no significant difference between 
the students who used COCA (M=2.92, SD=1.38) and those who used Tradooit 
(M=2.90, SD=.99), t(21)=.04, p=.96, suggesting that they improved to similar 
degrees. 


Table 1. Mean recognition scores 


Mean scores /10 (SD) 

Pre Post Delayed 
COCA 6.7 (1.0) 9.6 (.65) 9.8 (.37) 
Tradooit 6.6 (.74) 9.5 (.67) 9.6 (.70) 


Results for the production test are summarized in Table 2. The differences were 
again significant for both COCA, #(12)=6.79, p<.001, and Tradooit, 4(9)=6.00, 
p=.002. These increases were also maintained in the delayed post-test. The score 
increase was non-significant between the two groups, #(21)=0.41, p=.69. 


Table 2. Mean production scores 


Mean scores /10 (SD) 

Pre Post Delayed 
COCA 7.08 (1.0) 8.54 (.88) 8.78 (1.3) 
Tradooit 6.8 (1.0) 8.5 (1.3) 8.5 (1.27) 


3.2. Learners’ perceptions 


Learners’ ratings for ease of use, immediate usefulness (for the course), and general 
usefulness are provided in Table 3. Independent-samples ¢-tests revealed significant 
differences between the groups’ ratings for ease of use, (21)=2.59, p=.02, and 
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general usefulness, (21)=2.69, p=.01. There was no significant difference for 
immediate usefulness, 4(21)=1.62, p=.12. Tradooit users often commented on its 
simple interface (e.g. “it looks like a dictionary’’), while those assigned to COCA 
seemed intimidated by both the less-intuitive design and technical terms (e.g. 
matching strings). Both groups, however, commented that their concordancer 
helped with academic writing, an important component of the course. 


Table 3. Learners’ perceptions 


Ease of use Immediate Usefulness | General Usefulness 
COCA 7.62 (2.26) 6.69 (2.46) 5.69 (3.12) 
Tradooit 9.63 (.94) 8.5 (1.84) 8.56 (1.57) 


4. Conclusions 


We start our discussion with perhaps the most interesting result: the parallel 
concordancer was easier and more useful, although both types serve the immediate 
goal of improving academic writing. The interface issues addressed in the comments 
may have affected perceptions of general versus immediate usefulness: outside 
of coursework, students may be more inclined to resolve L2 issues by looking 
up translations or comparing expressions than by analyzing L2 samples. The 
similarity of immediate usefulness scores may also be due to the course material; 
during the concordancing assignments, students were likely picking up on features 
they could use in their writing. In other words, they may find concordancing more 
useful for features of immediate relevance than for general vocabulary or grammar 
development. 


The comparable learning gains suggest that the two concordancers might 
nevertheless offer the same benefits, but through different degrees of noticing 
and cognitive involvement. Tradooit, with direct L1-L2 comparisons, may entail 
less cognitive work but highlight subtle differences between the L1 and L2 (as is 
the case with many verb-preposition collocations), as proposed by Moon and Oh 
(2018). COCA may be more cognitively challenging, as learners had to analyze 
concordance lines, but consistent with Smart (2014), this may have resulted in 
awareness and learning of the target features. Of course, because this study was 
concerned with learning outcomes and perceptions, these possibilities should 
be tested in future research. Process-focused procedures such as observation, 
stimulated recall, or mouse-tracking can offer further insight on learners’ actual 
use of the concordancer and what specific features they find useful. 
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